Digital image classification method for cervical fluid-based cells based on a deep learning detection model

ABSTRACT

The present invention relates to the field of medical technology, and more particularly, to a digital image classification method for cervical fluid-based cells based on a deep learning detection model. The method comprises the following steps: selecting and labeling positions and categories of abnormal cells or biological pathogens in a digital image of cervical liquid-based smears; performing data normalization processing on the digital image of the cervical liquid-based smears; performing model training to obtain a trained Faster-RCNN model by taking the normalized digital image of the cervical liquid-based smears as an input, and the labeled position and category of each abnormal cell or biological pathogen as an output; and inputting an image to be recognized into the trained model and outputting a classification result. The method provided by the embodiment of the present invention can achieve the following advantages: abnormal cells or biological pathogens in a cervical cytological image are positioned; the abnormal cells or biological pathogens in the cervical cytological image are classified; and slice-level diagnostic recommendations are derived by recognizing the positioned abnormal cells or biological pathogens.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese Patent Application No.2020109677709 filed on Sep. 15, 2020, the entire disclosure of which ishereby incorporated by reference for all proper purposes.

TECHNICAL FIELD

The present invention relates to the field of medical technology, andmore particularly, to a digital image classification method for cervicalfluid-based cells based on a deep learning detection model.

BACKGROUND ART

Cervical cancer is the fourth most common cause of cancer death inwomen. Early detection and timely treatment of a disease can greatlyincrease a cure rate. Therefore, the early diagnosis of cervical canceris of great significance to women's health.

Cervical thin prep liquid-based cytology test (TCT) is a cervicalcytological diagnosis method which is often used to detect cervicalcancer and other diseases. With the development of medical digitization,modern hospitals are gradually advancing the diagnosis by examiningliquid-based cell smear images on a computer, instead of the traditionalmethod of directly observing and diagnosing liquid-based cell smearsunder a microscope. Digital images have the advantages of easy storageand management, easy transmission and consultation, easy retrospectivevisits, and relatively low cost of use.

The examination steps for diagnosis on the computer are as follows:

1, a sample of a small amount of cervical cells is first taken to make acervical liquid-based smear;

2, the cell smear is then scanned with a high-resolution scanner togenerate a digital image of the cervical liquid-based smear; and

3, finally, doctors and pathologists observe whether the cells areabnormal and make a diagnosis by using a computer image reading tool.

The cervical liquid-based smear usually contains 5,000-20,000well-preserved squamous cells or squamous metaplastic cells. In thediagnosis process, the recognition and positioning of abnormal cells orbiological pathogens will play a very important role in distinguishingand diagnosing diseased cells and reducing the burden of doctors inscreening. Therefore, a cell image automatic positioning and recognitiontechnology will be of great help to computer-aided screening andauxiliary diagnosis.

In view of the recognition and positioning of abnormal cells orbiological pathogens in the published patents, the following technicalsolutions are described.

Patent CN108364032A proposed a cervical cancer cell photo recognitionalgorithm based on a convolutional neural network. This method comprisesthe following steps: positioning cell nuclei with a watershed algorithmand segmenting a cell photo with the cell nuclei as centers; thenclassifying the segmented images by using a LeNet5 convolutional neuralnetwork to obtain classification results of the corresponding cells.This patent only involves classifying manually designated cells, butcannot automatically process a digital image of the entire cervicalliquid-based smear.

Patent CN109087283A proposes a method for recognizing diseased cells ina cervical cytopathological slice based on cell clusters. This methodcomprises the following steps: obtaining a foreground image of cellclusters through binarization processing by taking the cell clusters asa processing and recognition unit; and then performing cell clusterclassification on the extracted foreground through a classificationmodel of a deep convolutional neural network.

Patent CN109190567A proposed an automatic detection method for abnormalcervical cells based on a deep convolutional neural network. This methodis mainly characterized by classifying negative cells (normal cells) ina digital image, while only classifying positive cells into a singlecategory of “positive cervical cells”. This patent did not involve adetailed classification of positive cells.

Patent CN110163102A proposed a cervical cell image classification andrecognition method based on a convolutional neural network. This methodcomprises the following steps: segmenting an image into nucleus regionsto be detected; and then classifying the segmented nucleus regions byusing a dense convolutional network to obtain the categories of cells.This patent did not clearly describe the used image segmentation method,and the used network is a classification network without a positioningfunction.

The positioning technologies described in the above-mentioned patentshave insufficient classification accuracy, and in particular, aredifficult to have satisfactory fault tolerance for different sliceproduction methods, such that the sensitivity and specificity of theoverall slice-level results are also obviously insufficient.

SUMMARY OF THE INVENTION

In view of the foregoing technical problems, an embodiment of thepresent invention provides a digital image classification method forcervical fluid-based cells based on a deep learning detection model tosolve one or more problems of inaccurate recognition and positioning ofabnormal cells or biological pathogens, and low fault tolerance.

In a first aspect of an embodiment of the present invention, there isprovided a digital image classification method for cervical fluid-basedcells based on a deep learning detection model, which comprises thefollowing steps: a data preparation phase: selecting and markingpositions and categories of abnormal cells or biological pathogens in adigital image of cervical liquid-based smears; a data processing phase:performing data normalization processing on the digital image of thecervical liquid-based smears; a model training phase: performing modeltraining to obtain a trained Faster-RCNN model by taking the normalizeddigital image of the cervical liquid-based smears as an input, and thelabeled position and category of each abnormal cell or biologicalpathogen in the digital image of the cervical liquid-based smears as anoutput; and an output phase: inputting an image to be recognized intothe trained Faster-RCNN model and outputting a classification result.

Optionally, the step of labeling the positions and categories of theabnormal cells or biological pathogens in the digital image of thecervical liquid-based smears specifically comprises: selecting a labeledregion in each digital image of the cervical smears; performingrectangular region labeling on abnormal cells or biological pathogens inthe labeled region; and recording coordinate positions of upper left andlower right vertices of each rectangle in the rectangle region labeling,and storing the categories of the abnormal cells or the biologicalpathogens corresponding to the rectangle.

Optionally, a profile of the rectangular region labeling completelycovers the region of the abnormal cells or the biological pathogens.

Optionally, the step of performing data normalization processing on thedigital image of the cervical liquid-based smears specificallycomprises: reading a pixel parameter of each digital image of thecervical liquid-based smears, where the pixel parameter represents anactual distance between each pixel and its corresponding cervical smear;and zooming in and out the digital images of the cervical smearsaccording to the image parameters to realize the normalization ofphysical dimensions.

Optionally, the pixel parameter is 0.5, and a formula for zooming in andout the digital image of the cervical smears is as follows: the numberof pixels in the target line=0.5*the number of pixels in the originalline; and the number of pixels in the target column=0.5*the number ofpixels in the original column.

Optionally, the method further comprises: performing a flip and/ormirroring operation on the selected digital image of the cervical smearsto expand a data set.

Optionally, the output result is predicted probabilities respectivelycorresponding to the case that the target is a background, abnormalcells or biological pathogens.

Optionally, the model parameters are obtained by training in the modeltraining phase by means of a backpropagation algorithm.

Optionally, the method further comprises: setting a confidencethreshold, and displaying a prediction result according to theconfidence and calculation rules.

Optionally, the confidence is calculated using the following formula:

V=e ^(−x) ² ^(ln2)

wherein, V is the confidence, e is a natural constant, x is a thresholdratio, and In is a logarithm based on a natural constant.

According to the digital image classification method for cervicalfluid-based cells based on the deep learning detection model provided bythe embodiment of the present invention, first of all, the faulttolerance of the classification method of the present invention fordifferent slice production methods is improved by using the highlyefficient feature extraction capabilities and the diverse training datasets of the deep neural network; secondly, the positioning accuracy isgreatly improved by a region proposal network scheme in the targetdetection model of the deep convolutional neural network; thirdly, theaccuracy of the classification of abnormal cells or biological pathogensis effectively improved by a classification network scheme in the targetdetection model of the deep convolutional neural network; and finally, awhole-slice diagnosis suggestion is obtained with higher sensitivity andspecificity by using the rules and formulas designed in the presentinvention. The digital image classification method for cervicalfluid-based cells based on the deep learning detection model provided bythe embodiment of the present invention can achieve the followingadvantages: abnormal cells or biological pathogens in a cervicalcytological image are positioned; the abnormal cells or biologicalpathogens in the cervical cytological image are classified; andslice-level diagnostic recommendations are derived by recognizing thepositioned abnormal cells or biological pathogens.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments are exemplified by the photos in thecorresponding accompanying drawings, and these exemplified descriptionsdo not constitute a limitation on the embodiments. Components with thesame reference numerals in the accompanying drawings represent similarcomponents. Unless otherwise stated, the figures in the accompanyingdrawings do not constitute a limitation of scale.

FIG. 1 is a flowchart of a digital image classification method forcervical fluid-based cells based on a deep learning detection modelaccording to an embodiment of the present invention;

FIG. 2 is a digital image of cervical liquid-based smears that meetclinical standards as provided by an embodiment of the presentinvention; and

FIG. 3 is a digital image of cervical liquid-based smears in a labeledregion as provided by an embodiment of the present invention.

DETAILED DESCRIPTION

In order to facilitate the understanding of the present invention, thepresent invention will be further described in detail with reference toaccompanying drawings and specific embodiments. It should be also notedthat when a component is referred to as “being fixed to” the othercomponent, the component can be directly disposed on the othercomponent, or there may be one or more intermediate components locatedtherebetween. When a component is referred to as “being connected with”the other component, the component can be directly connected to theother component, or there may be one or more intermediate componentslocated therebetween. The orientation or positional relationshipsindicated by the terms “upper”, “lower”, “inner”, “outer”, etc. areorientation or positional relationships shown on the basis of theaccompanying drawings, only for the purposes of the ease in describingthe present invention and simplification of its descriptions, but notindicating or implying that the specified device or element has to bespecifically located, and structured and operated in a specificdirection, and therefore, should not be understood as limitations to thepresent invention. The term “first”, “second”, “third” and the like usedare merely used to describe but not denote or imply any relativeimportance.

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by those skilled in the artto which the present invention belongs. The terms used herein in thedescription of the present invention are for the purpose of describingparticular embodiments only and are not intended to limit the presentinvention. The term “and/or” as used herein includes any and allcombinations of one or more of the associated listed items.

A deep convolutional neural network has developed rapidly in the fieldof machine vision in recent years to continuously refresh a number ofevaluation records in the academic world, such as the ImageNet Challengeand the MS-COCO Challenge, and thus had a profound impact on theindustry. The present invention realizes automatic detection andpositioning of various cells in cervical liquid-based cytological imagesand obtains slice-level diagnosis suggestions on the basis of a targetdetection model Faster-RCNN of a customized deep convolutional neuralnetwork. Meanwhile, according to the following embodiments of thepresent invention, the model is obtained by deep learning and finitelabeled data based on numerical methods, without manually designingfeatures. The data sets constructed by the method of the presentinvention can also include digital images of different slice productionmethods and scanning imaging parameters, so that the method of thepresent invention has stronger versatility and robustness for sliceproduction methods and imaging parameters. The present invention will bedescribed in detail below.

Referring to FIG. 1, an embodiment of the present invention firstprovides a digital image classification method for cervical fluid-basedcells based on a deep learning detection model. As shown in FIG. 1, themethod comprises the following steps.

Step 101, a data preparation phase: selecting and labeling positions andcategories of abnormal cells or biological pathogens in a digital imageof cervical liquid-based smears.

According to the method of the present invention, a convolutional neuralnetwork for image classification is constructed based on deep learning.Deep learning is a technology in which feature extraction and modelparameter adjustment are performed on the basis of a large number ofsamples through a backpropagation algorithm. In order to solve theproblem of positioning and classification of abnormal cells orbiological pathogens, in the data preparation phase of the method of thepresent invention, a digital image containing tens of thousands ofcervical liquid-based smears is first constructed, and positions andcategories of abnormal cells or biological pathogen nuclei in thedigital image of the cervical liquid-based smears are then labelled,which are specifically described as follows:

first of all, digital images of cervical liquid-based smears that meetclinical standards are selected, and a different number of regionscontaining cells with widths and heights of 1200 pixels are selectedfrom each image at a 20× lens resolution; and

the selected regions form labeled regions (21 in FIG. 2). The purpose ofselecting the labeled regions in each image is to make a target regionfully labeled, while avoiding over-labeling uninteresting regions,thereby saving the manpower and improving the efficiency.

Then, during the labeling process, rectangular labeling and recording ofabnormal cells or biological pathogens in the labeled region (22 in FIG.2) must satisfy: a profile labeled by the rectangle must completelycover the cell or biological pathogen region, while coordinates of theupper left and lower right vertices of each rectangle in the labeledregion need to be completely recorded, and the categories of theabnormal cells or biological pathogens corresponding to the rectangleare stored.

In an embodiment of the present invention, the categories of abnormalcells or biological pathogens that need to be labeled are as follows:

squamous cells include: atypical squamous epithelial cells (low-gradesquamous epithelial lesions, not excluding high-grade squamousepithelial lesions) and squamous cell carcinoma (high-grade squamousepithelial lesions);

glandular cells include: atypical glandular cells (cervical canal cells,endometrial cells), cervical canal glandular cells (prone to betumorous), cervical canal adenocarcinoma in situ, adenocarcinoma(cervical canal adenocarcinoma, endometrial adenocarcinoma, extrauterineadenocarcinoma);

biological pathogens include: Trichomonas vaginalis, fungi withmorphology consistent with Candida albicans (bacterial vaginosis issuggested in the case of dysbacteriosis), and bacteria with morphologyconsistent with actinomycetes (cytological changes accord with herpessimplex virus infection);

and endometrial cells.

Step 102, a data processing phase: performing data normalizationprocessing on the digital images of the cervical liquid-based smears.

The images selected in step 101 are all sampled from different digitalimages of cervical liquid-based smears, and these digital images may bescanned and imaged by different scanners. Therefore, the collectedimages needs to be normalized, due to the difference in hardwareattributes and software parameter settings of different scanners, andthe difference in actual physical dimensions represented by pixels ofeach image. The purpose of normalization is to ensure that the images ina data set have similar physical dimensions as much as possible. In thedeployment and application scenarios of the following deep convolutionalneural network model, input data should also be normalized in the sameway.

According to an embodiment of the present invention, a micron per pixel(mpp) parameter of an image can be obtained by reading additionalinformation of the image. As a pixel parameter, mpp represents an actualdistance of a cervical smear corresponding to each pixel, and mpp of 1represents that an actual horizontal or longitudinal distancerepresented by each pixel is 1 micron.

By reading the mpp, the images in the digital image data set of thecervical smears can be zoomed in or out by bilinear interpolation toachieve the normalization of physical dimensions of the data.

In the method of the present invention, the data set mpp is normalizedto 0.5. The number of pixels in a target line (column) of each photo iscalculated by the following formula:

the number of pixels in the target line (column)=0.5*the number ofpixels in the original line (column)/mpp.

Step 103, a model training phase: performing model training to obtain atrained model by taking the normalized digital image of the cervicalliquid-based smears as an input, and the labeled position and categoryof each abnormal cell or biological pathogen in the digital image of thecervical liquid-based smears as an output.

In this step, the Faster-RCNN deep convolutional neural network model istrained to obtain the trained network model by taking the correspondingcoordinates and category corresponding to rectangular region labeling ofeach abnormal cell or the biological pathogen in the image as theoutput.

In the embodiment of the present invention, in order to use the finitetraining data to make the generalization ability of the model stronger,the digital images of the cervical smears in the training set can beflipped and/or mirrored to realize the expansion of the data set. Forease of description, a data set composed of originally selected digitalimages of cervical smears is named a training set (1) below. Taking thetraining set (1) as basic data, the specific steps for data setexpansion are as follows:

mirroring: the training data set (1) and labeled images thereof aremirrored vertically or horizontally at the same time, and are thenaggregated with the training data set (1) to form a training data set(2).

flip: the training data set (2) and labeled images thereof are flippedclockwise at the same time, at a flip angle of 90 degrees, 180 degreesor 270 degrees, and are then aggregated with the training data set (2)to form a training data set (3). The expanded training data set (3)includes training data used to train a neural network.

The expanded training data set (3) is input into the Faster-RCNN deepconvolutional neural network model of the present invention fortraining. When the deep convolutional neural network model is trained, adigital image of cervical liquid-based smears of 1200×1200 pixels istaken as an input of the model, and coordinates and a category label (31in FIG. 3) of a label box of each abnormal cell or biological pathogenin the image is taken as an output of the model. It should be noted thatthe above-mentioned mirroring followed by flipping is only one of theembodiments of the present invention to expand the training set. Inother embodiments, the training set may also be expanded by flippingfollowed by mirroring, or only by flipping, or only by mirroring.

The training model used in the method of the present invention is theFaster-RCNN model. The Faster-RCNN model is an image segmentation modelbased on a convolutional neural network. It is unnecessary to manuallydesign features for this model. A large amount of labeled data can beused to train the model to obtain a good positioning and classificationeffect. In the embodiment of the present invention, training the modelincludes the following steps.

1. Feature Extraction Network

In the training process, a digital image of cervical liquid-based smearsof 1200*1200 pixels is input into the deep convolutional neural networkfor feature extraction. The feature extraction network consists ofrepeatedly stacked convolutional layers, sampling layers and nonlinearactivation layers. This neural network architecture summarizes andextracts abstract features of an image by pre-training with a largeamount of image data and category labels of objects contained in theimages on the basis of the backpropagation algorithm in deep learning,and outputs a high-dimensional feature tensor of the image.

The feature extraction network applied in the embodiment of the presentinvention is a feature extraction network of a modified Resnet-101classification network. A network architecture diagram of the featureextraction network is shown in Table 1, where there is no non-linearactivation layer between respective cycles. A 1200*1200 image is takenas an input of the feature extraction network, and four high-dimensionaltensors of 300*300*2048, 150*150*2048, 75*75*2048, 38*38*2048 are takenas an output of the feature extraction network.

TABLE 1 Input Size of Step Module Name of layer dimension computing corelength cycle Input 1200*1200*3 — — Convolution 1 600*600*64 7*7 2 Layer1.0. Pooling 300*300*256 1*1 2 Layer 1.0. Convolution 1 300*300*64 1*1 1 3 cycles Layer 1.0. Convolution 2 300*300*64 3*3 1 Layer 1.0.Convolution 3 300*300*256 1*1 1 Layer 2.0. Pooling 150*150*512 1*1 2Layer 2.0. Convolution 1 150*150*128 1*1 1  4 cycles Layer 2.0.Convolution 2 150*150*128 3*3 1 Layer 2.0. Convolution 3 150*150*512 1*11 Layer 3.0. Pooling 75*75*1024 1*1 2 Layer 3.0. Convolution 1 75*75*2561*1 1 23 cycles Layer 3.0. Convolution 2 75*75*256 3*3 1 Layer 3.0.Convolution 3 75*75*1024 1*1 1 Layer 4.0. Pooling 38*38*2048 1*1 2 Layer4.0. Convolution 1 38*38*512 1*1 1  3 cycles Layer 4.0. Convolution 238*38*512 3*3 1 Layer 4.0. Convolution 3 38*38*2048 1*1 1

2. Region Propose Network

Then, the extracted deep convolution features are input into a regionproposal network. The region proposal network is composed of fullyconnected layers and nonlinear activation layers. The region proposalnetwork performs sliding window classification and object bounding boxcoordinate regression on the high-dimensional tensors output by thefeature extraction network. The classification result refers todetermining a probability that the current window position containsabnormal cells or biological pathogens and estimating dimensions andaspect ratios of the cells contained in the current window. The currentwindow position corresponds to the corresponding coordinate position inthe original image.

A network architecture diagram of the region proposal network is shownin Table 2. According to the embodiment of the present invention, a3*3*256 convolution and a sliding window along the first two dimensionson the corresponding four high-dimensional tensors are taken as an inputof the region proposal network. An intermediate layer is a256-dimensional feature vector. A classification output layer is thefully-connected layer. The fully-connected output of the 256-dimensionalfeature vector is the categories of objects included in the currentregion, wherein the vector [0,1] represents a background, the vector[1,0] represents abnormal cells or biological pathogens, and therectangular box position regression also occurs in the fully-connectedlayer. The fully-connected output of the 256-dimensional vector isfloating point values of the objects included in the current region in[0,1] relative to the normalization of the lengths and the widths inhorizontal and longitudinal directions of coordinates of the upper leftcorner of an external rectangular box in the center of sub-tensorcoordinates.

TABLE 2 Size of Name of layer Input dimension computing core Step lengthDescription fpn. inner layer 1 300*300*256 1*1 1 Pooling fpn. layer 1300*300*256 3*3 1 Convolution fpn. inner layer 2 150*150*256 1*1 1Pooling fpn. layer 2 150*150*256 3*3 1 Convolution fpn. inner layer 375*75*256 1*1 1 Pooling fpn. layer 3 75*75*256 3*3 1 Convolution fpn.inner layer 4 38*38*256 1*1 1 Pooling fpn. layer 4 38*38*256 3*3 1Convolution rpn. Pooling 7*7*256 3*3 1 Pooling rpn. Classification2*12544 1*1 1 Fully-connected layer rpn. Border prediction 8*12544 1*1 1Fully-connected layer

3. Classification Network

At last, the classification network classifies the high-dimensionalfeature tensors corresponding to positions containing abnormal cells orbiological pathogens output by the region proposal network, anddetermines that a target contained in this region is abnormal cells, ordetailed categories of biological pathogens or a background. Theclassification network is composed of stacked fully-connected layers andnonlinear activation layers. A network architecture diagram of theclassification network is shown in Table 3. The image scale of thefeature extraction network is reduced by 32 times from input to output.Therefore, the lengths and widths in the horizontal and longitudinaldirections output by the region proposal network need to be enlarged by32 times, which is the size of a detection box in the original image.

TABLE 3 Size of Name of layer Input dimension computing core Step lengthDescription roi. fully-connected layer 6 1024 — — roi. fully-connectedlayer 7 1024 — — roi. classification output  16 — — 16 represents 16categories of classification results

In the above-mentioned neural network architecture of the presentapplication, the nonlinear activation layer adopts a rectified linearunit (ReLU), and the convolutional layer and the fully-connected layerare followed by the nonlinear activation layer of the ReLU function. Theformula of ReLU is as follows, wherein max represents a maximum takenfrom two input numbers:

ReLU(x)=max(0,x)

According to the embodiment of the present invention, thebackpropagation algorithm in deep learning is used for training toobtain model parameters. The classification network and the regionproposal network take a target real category vector and the coordinatesof the input region relative to the coordinates of the center of theinput tensor as labels, and the loss function is a cross entropyfunction.

In the embodiment of the present invention, the parameters of thefeature extraction network are initialized by removing the parameters ofthe fully-connected layer from a network pre-trained in the ImageNetclassification network. Other relevant network parameters are randomlyinitially selected from parameters in [0,1] that obey the truncatednormal distribution. A stochastic gradient descent backpropagationalgorithm is used to train 90 cycles in an enhanced training set with alearning rate of 0.001.

After the above training is completed, segmentation results are countedthrough the obtained model on a verification set. That is, all thesegmentation results of each image in the verification set aresuperimposed together to form a segmentation template of this image.Finally, a Euclidean distance between the segmentation template and theactual label is calculated. The Euclidean distance is an inference errorof a single image. At last, the inference errors of all the images inthe verification set are added to obtain a verification set error. Inthe training process selected in the embodiment of the presentinvention, a model with the minimum verification set error is selectedas the Faster-RCNN model obtained by final training.

In the embodiment of the present invention, a region with a maximumpredicted probability higher than a threshold of 0.4 is regarded as thefinal output of the model. All targets output by the model are processedwith a non-maximum suppression (NMS) algorithm to eliminate highlyoverlapping detection results and regarded as the final output of thealgorithm. In the present invention, redundant calculations in imageoperations can be reduced by adaptive thresholds and the NMS algorithm,thereby achieving a huge improvement in image processing efficiency.

Step 104, an output phase: inputting an image to be recognized into thetrained Faster-RCNN deep convolutional neural network model andoutputting a classification result.

In application, the output of the classification network is a predictedprobability that this target region is a background, abnormal cells orbiological pathogens. In this step, a digital image of cervicalliquid-based cells to be recognized needs to be input into the trainedFaster-RCNN deep convolutional neural network model. By means of theabove-mentioned feature extraction, the selection of the region proposalnetwork and final classification, different numbers of abnormal cells orbiological pathogens and their corresponding predicted probabilities areobtained.

According to the digital image classification method for cervicalfluid-based cells based on the deep learning detection model provided bythe embodiment of the present invention, any digital image of cervicalliquid-based smears is inputted into the Faster-RCNN model obtained inStep 103 to obtain whether the target is abnormal cells, the detailedcategories of biological pathogens, or the background. It should benoted that the model training method in the embodiment of the presentinvention is as a result of the creative work of those skilled in theart. Any change, adjustment, or replacement scheme for the dataenhancement method, neural network architectures, hyperparameters, andloss function in the present invention on the basis of the embodimentsof the present invention should be regarded as being equivalent to thissolution.

After the predicted probability that the target is abnormal cells, thedetailed categories of biological pathogens, or the background isobtained, the user can set a confidence threshold to display a predictedresult that is greater than the set confidence threshold.

In the present invention, a confidence calculation method is as follows:

first of all, for each category without background, 16 detection resultswith the maximum probabilities are selected as basic data for thecalculation of smear results in the embodiment of the present invention(all detection results are used as the basic data for the calculation ofthe smear results if there are less than 16 detection results).

For each category, the selected basic data are averaged to obtain ascore of this category, and a score-to-threshold ratio of this categoryis obtained by dividing the score by the corresponding threshold.Through the score-to-threshold ratio, the confidence of slice-levelresults is then obtained by the following formula:

V=e ^(−x) ² ^(ln2)

-   -   where V is the confidence, e is a natural constant, x is a        score-to-threshold ratio, and In is a logarithm based on a        natural constant.

After the confidence of each category is obtained, in the embodiment ofthe present invention, the confidence of the category is considered as apotential category if it exceeds 0.5.

In the embodiment of the present invention, the potential category ofabnormal cells will obtain a final slice-level result according to thefollowing priority. If there is no potential category, the result isnegative:

atypical glandular cells (including cervical canal and endometrialcells)>high-grade squamous epithelial lesions>atypical squamousepithelial cells (not excluding high-grade squamous epitheliallesions)>low-grade squamous epithelial lesions>atypical squamousepithelial cells (undefined).

The potential categories of biological pathogens will all be listed asresults. If there are no potential categories, the result will benegative.

After the confidence of each category of cells is obtained, it isnecessary to further rank the confidence of each category of abnormalcells in a descending order. The positive confidence in each smear canbe obtained by continuously applying the following formulas in pairssuccessively:

$\coprod{\left( {V_{1},V_{2}} \right)\left\{ \begin{matrix}{{\frac{V_{1} + V_{2}}{1 + {4V_{1}V_{2}}};\ {{if}\mspace{14mu} V_{1}}},{V_{2} < {0.5}}} \\{{\frac{V_{1} + \left( {V_{2} - {0.5}} \right)}{1 + {V_{1}\left( {V_{2} - {0.5}} \right)}};\ {{if}\mspace{20mu} V_{1}}},{V_{2} > {0.5}}} \\{V_{1};\ {{other}\mspace{14mu}{cases}}}\end{matrix} \right.}$

in which, V1 and V2 are two confidences entered in the applying process.For example, for the existing confidences V1, V2, V3, and V4 ranked in adescending order, the formula will be applied three times in successionwhen calculating the confidence, as shown in the following formula:

Confidence=

(

(

(V ₁ ,V ₂),V ₃),V ₄)

In the embodiment of the present invention, first of all, the faulttolerance of the classification method of the present invention fordifferent slice production methods is improved by using the highlyefficient feature extraction capabilities and the diverse training datasets of the deep neural network; secondly, the positioning accuracy isgreatly improved by the region proposal network (RPN) scheme in thetarget detection model of the deep convolutional neural network;thirdly, the accuracy of the classification of abnormal cells orbiological pathogens is effectively improved by a classification networkscheme in the target detection model of the deep convolutional neuralnetwork; and finally, a whole-slice diagnosis suggestion is obtainedwith higher sensitivity and specificity by using the rules and formulasdesigned in the present invention. That is, the digital imageclassification method for cervical fluid-based cells based on the deeplearning detection model provided by the embodiment of the presentinvention can achieve the following advantages:

1. abnormal cells or biological pathogens in a cervical cytologicalimage are positioned;

2. the abnormal cells or biological pathogens in the cervicalcytological image are classified; and

3. slice-level diagnostic recommendations are derived by recognizing thepositioned abnormal cells or biological pathogens.

Compared with Patent CN108364032A, the method of the present inventioncan not only automatically position abnormal cells that need to beclassified, but also position and recognize biological pathogens.

Compared with Patent CN109087283A, the method of the present inventioncan not only recognize abnormal cells and biological pathogens in cellclusters, but also can recognize and position discrete cells with highaccuracy. In addition, the pre-processing process is more concise andfaster.

Compared with the patent CN109190567A, the method of the presentinvention can effectively classify positive abnormal cells andbiological pathogens in detail.

Compared with the patent CN110163102A, the method of the presentinvention adopts automatic effective region segmentation, and automaticpositioning of abnormal cells and pathogenic microorganisms, therebysaving a lot of manpower.

Therefore, compared with the above four patents, the method of thepresent invention can not only realize the positioning andclassification of abnormal cells or biological pathogens on the digitalimages of the cervical liquid-based smears, but also can give diagnosissuggestions on the slice-level results, which play an auxiliary role forclinicians, thereby reducing the workload of doctors. At the same time,the slice-level diagnosis suggestions given by the present inventionhave higher sensitivity and specificity. It should be noted that themethod of the present invention can also be applied to automaticdetection of other pathological digital images in the medical field,such as the detection of exfoliated cells of urine, which will not belimited in the present invention.

It should be further appreciated by those skilled in the art that,various steps of the exemplary bifocal image integration methoddescribed in conjunction with the embodiments disclosed herein may beimplemented as electronic hardware, computer software or a combinationthereof. In order to clearly illustrate the interchangeability betweenthe hardware and the software, the constitution and steps of variousexamples are described generally according to the functions in the abovedescription. Whether these functions are implemented as hardware orsoftware depends on particular applications and design constraints oftechnical solutions.

Those skilled in the art may implement the described functions withdifferent methods for each of particular applications, but suchimplementation shall not be regarded as going beyond the scope of thepresent invention. The computer software may be stored in a computerreadable storage medium, and when this program is executed, theprocesses of the above-mentioned method embodiments will be included.The storage medium may be a magnetic disk, an optical disc, a read-onlystorage memory, a random storage memory, or the like.

It should be eventually noted that: the above embodiments are only usedto illustrate the technical solutions of the present invention, ratherthan limiting these technical solutions; under the concept of thepresent invention, the technical features in the above embodiments ordifferent embodiments can also be combined, the steps can be implementedin any order, and there are many other variations of different aspectsof the present invention as described above, which are not provided inthe details for the sake of clarity. Although the present invention hasbeen described in detail with reference to the foregoing embodiments,those of ordinary skill in the art should understand: it is stillpossible to modify the technical solutions described in the foregoingembodiments, or equivalently replace some of the technical features.However, these modifications or replacements do not cause the essence ofthe corresponding technical solutions to deviate from the scope of thetechnical solutions of the embodiments of the present invention.

What is claimed is:
 1. A digital image classification method forcervical fluid-based cells based on a deep learning detection model,comprising the following steps: a data preparation phase: selecting andlabeling positions and categories of abnormal cells or biologicalpathogens in a digital image of cervical liquid-based smears; a dataprocessing phase: performing data normalization processing on thedigital image of the cervical liquid-based smears; a model trainingphase: performing model training to obtain a trained Faster-RCNN modelby taking the normalized digital image of the cervical liquid-basedsmears as an input, and the labeled position and category of eachabnormal cell or biological pathogen in the digital image of thecervical liquid-based smears as an output; and an output phase:inputting an image to be recognized into the trained Faster-RCNN modeland outputting a classification result.
 2. The method as claimed inclaim 1, wherein the step of labeling the positions and categories ofthe abnormal cells or biological pathogens in the digital image of thecervical liquid-based smears specifically comprises: selecting a labeledregion in each digital image of the cervical smears; performingrectangular region labeling on the abnormal cells or biologicalpathogens in the labeled region; and recording coordinate positions ofupper left and lower right vertices of each rectangle in the rectangleregion labeling, and storing the categories of the abnormal cells or thebiological pathogens corresponding to the rectangle.
 3. The method asclaimed in claim 2, wherein a profile of the rectangular region labelingcompletely covers the region of the abnormal cells or the biologicalpathogens.
 4. The method as claimed in claim 1, wherein the step ofperforming data normalization processing on the digital image of thecervical liquid-based smears specifically comprises: reading a pixelparameter of each digital image of the cervical liquid-based smears,where the pixel parameter represents an actual distance between eachpixel and its corresponding cervical smear; and zooming in and out thedigital images of the cervical smears according to the image parametersto realize the normalization of physical dimensions.
 5. The method asclaimed in claim 4, wherein the pixel parameter is 0.5, and a formulafor zooming in and out the digital images of the cervical smears is asfollows: the number of pixels in the target line=0.5*the number ofpixels in the original line; and the number of pixels in the targetcolumn=0.5*the number of pixels in the original column.
 6. The methodaccording to claim 1, further comprising: performing a flip and/ormirroring operation on the selected digital images of the cervicalsmears to expand a data set.
 7. The method as claimed in claim 2,wherein the output result is predicted probabilities respectivelycorresponding to the case that the target is a background, abnormalcells or biological pathogens.
 8. The method as claimed in claim 1,wherein model parameters are obtained by training in the model trainingphase by means of a backpropagation algorithm.
 9. The method as claimedin claim 1, further comprising: setting a confidence threshold, anddisplaying a prediction result according to the confidence andcalculation rules.
 10. The method as claimed in claim 9, wherein theconfidence is calculated using the following formula:V=e ^(−x) ² ^(ln2) wherein, V is the confidence, e is a naturalconstant, x is a score-to-threshold ratio, and In is a logarithm basedon a natural constant.